Montgomery reduction
The Montgomery representation of $x \in [0, N-1]$ is $\overline{x} = xR \ mod \ N$
$N$: an n-word integer
$R$: some integer greater than $N$ and co-prime with $N$, $gcd(R, N)=1$
- for a efficient algorithm, $R$ is always a power of the radix $b$, for example, $R = b^n$
Montgomery reduction is used to speed up modular multiplication.
Assume 2 integers $a$ and $b$, the goal is to compute $c = ab \ mod \ N$.
Select $R \in \Bbb{N}$ such that $R > N$ and $gcd(R, N) = 1$
First, transfer $a$ and $b$ to Montgomery form
$\overline{a} = aR \ mod \ N$
$\overline{b} = bR \ mod \ N$
Then compute $c = ab \ mod \ N$ from $\overline{a}$ and $\overline{b}$
where $R^{-1}$ such that $RR^{-1} \equiv 1 \ mod \ N$
Montgomery reduction - REDC algorithm
The Montgomery reduction of $u \in [0, RN-1]$ is $Redc(u) = uR^{-1} \ mod \ N$
So the computation from $\overline{a}$ and $\overline{b}$ to $c = ab \ mod \ N$ can be rewrite as
$\overline{c} = Redc(\overline{a} \overline{b})$
$c = Redc(\overline{c})$
The naive operation $uR^{-1} \ mod \ N$ need to multiply by $R^{-1}$ and divide by $N$. The problem is that the division by $N$ is expensive on most computer hardware.
The major idea of REDC is try to avoid dividing by $N$ but by $R$. As $R$ can be chosen to be a power of 2. The division of $2^n$ can be replaced by shifting which are fast.
The we can find a integer $\hat{s}$ such that $\hat{u}R = u + \hat{s}N$.
How to find the integer $\hat{s}$?
Try
Replace $\hat{s}$ with $s$.
So $u + sN$ is a multiple of $R$ and $u + sN \equiv u \ mod \ N$.
Finally
If $0 \le v < n$ then $\hat{u} = Redc(u) = v$. Otherwise $\hat{u} = Redc(u) = v - n$
refer
https://www.nayuki.io/page/montgomery-reduction-algorithm
https://en.wikipedia.org/wiki/Montgomerymodularmultiplication